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A PSTP ACT 

The major problem investigated was the effectiveness 
of cloze procedure as a predictor of a student’s ability to 
comprehend social studies materials when compared with I.Q. scores, 
previous social studies grades, and standardized reading test scores. 
As a secondary purpose, the effectiveness of rewritten social studies 
materials as a means of improving comprehension was studied. As a 
preliminary, various readibility measures formulas were reviewed. 
Those included were: Lively and Pressey, Gray and Leary, Lorge, 
Flesch, Dale-Chall. Research on the application and effect on 
comprehension of these formulas was also reviewed. Students completed 
a pre-reading cloze test from one of two texts utilized. They then 
read the chapter from which the cloze test had been constructed, and 
completed a fifty item multiple- choice test. The cloze procedure was 
found not to be better than the other variables in predicting _ 
comprehension levels at the .01 significance level. However, it was 
significant at the .05 level. To fulfill the secondary purpose, two 
identical texts were used, however, the readability levels were 
different (grade 5-6, grade 7-B) . A multiple-choice test was 
constructed to measure knowledge acquired after reading. ^ It was found 
that reducing vocabulary difficulty and sentence complexity may not 
significantly improve comprehension scores. (SBE) 
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CLOZE PROCEDURE AS A PREDICTOR OF COMPREHENSION IN 
SECONDARY SOCIAL STUDIES MATERIALS 



BTTRODUCTION 

The investigation upon which this presentation is based involved the 
following problems; 

1. The major problem investigated how predictive of a student's 
ability to comprehend social studies materials are cloze 
procedure scores when compared ivith I.Q. scores, previous social 
studies grades, and standardized reading test scores. 

2. The secondary investigation dealt with the problem, does 
rewritten social studies materials on an easier readability 
level improve the comprehension of that material? 




Development of Readability Measures 
Interest in assessing printed materials has existed for some time. 

Lorge^ indicates that the Talmudists in A.D. 900 counted words in a usual 
or unusual sense. One of the first scientifically oriented attempts to 
quantify a readability factor occurred in 1889 when F. ¥. Kaeding attempted 
to ascertain the frequency of occurrence of 11,000,000 words. The importance 
of the above study along with Thorndike's investigation^ or word frequency 
^ is suggested by the initial inclusion of vocabulary factors alone in the 
Lively arii Pressey readability formula.^ This formula is credited by 
Chall^ as being the first quantitative study of readability. 
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By 1928 the emphasis on vocabulary factors as the basis of predicting 
readability was recognized as being inadequate. During the second period 
of readability e:xploration, extending through 193 9^ investigators of 
readability searched for factors other than vocabulary which would provide 
more accuracy in prediction. Representative of this period is the work 
of Gray and Leary. 2 In studying previous findings in readability and 
securing the opinions of about 100 experts and 170 library patrons, Gray 
and Leary found 3^9 factors which were assigned to th.e categories of 
content, style of expression and presentation, format, and general 
features of organization. 

Difficulties in evaluating qualitative factors and the interrelated- 
ness of many of the variables investigated by Gray and Leary were 
instrumental on ushering in the next period of readability investigation. 
During this period, which began about 1939 with the appearance of the 
Lorge Readability formula,^ the basis for development of readability 
formulas rested on the premise that a small number of factors could 
validly predict readability. The two-factor Flesch and Dale-Chall 
formulas were credited by Chall'^ with giving a readability prediction 
comparable to the five- factor Gray and Leary formula. 

Limitations of Readability Formulas 

In the process of objectifying and simplifying the application of 
readability formulas, a measure of vocabulary and sentence factors was 
usually included. A source of criticism of these formulas lies in their 
avoidance of measuring other factors of readability. 
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Lorge^ indicates that readability formulas measure four elements. 
They are vocabulary load, sentence structure, idea density, and human 
interest. He adds that no other internal elements of comprehensibility 
have been useful in predicting passage difficulty although the lack of 
a measurement of conceptual difficulties and organization of the printed 
material is a fundataental weakness of formulas. 

Chall^ adds reinforcement to the above statement. She suggests that 
readability formulas do not measure abstractness, vagueness, illogical 
organization, difficulty of words, conceptual difficulty, content, and 
physical features. 

Smith and Dechant^^ support the above statements while attending to 
certain variables not previously mentioned. They state that readability 
formulas pay little attention to six factors which are determinants of 
readability. These factors are density and unusualness of facts, number 
of pi ct oral illustrations, interest and purpose, concept load and 
abstractness of words, organization of material and format, and interre- 
lationship of ideas. 
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Dale and Chall suggest that three variables affect readability. 
Included are the printed material and its stylistic elements, the 
criterion measure and the method used to make the readability estimate, 
and the reader along lAdth the qualities he brings to the printed page. 

In summary, a limitation of the readability formulas appears to be 
evident with consideration of the variables mentioned above by Dale and 
Chall as only two of these factors are quantified. Since individual 
capabilities and characteristics are not considered in application of 
readability formulas for evaluation of xvritten materials, difficultiigs 
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may be encountered when one attempts to equate the reader and instructional 
material on the basis of such quantification. The discussion below 
expands and supports this statement. 

Readability Formula Application and Effect on Comprehension 

The statements above note certain limitations which are associated 

with quantitative evaluations of printed materials. Results of empiracal 

assessments of readability formula procedures are presented below. 

Since one of the elements common to the most widely used readability 

formulas is some measurement of vocabulary, this variable would logically 
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be included in investigations of readability assessment. Nolte 

investigated the effects of comprehension on mechanically simplifying 

vocabulary terms. Pictoral tests c.nd personal interviews were employed 

to measure comprehension. Nolte reported, "Many vocabulary difficulties 

and numerous erroneous concepts were disclosed... 
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Wilson's study included a three-hundred word passage which was 
amplified into six-hundred and twelve-hundred word versions. Since 
students comprehended the longest and structurally most difficult version 
significantly better, the efficacy of simplifying sentence factors as 
a means of improving comprehension may be open to question. 

14 

McC racken investigated the effectiveness of applying readability 
formula criteria in producing more readable materials. He rated the di 
difficulty of two passages by the Yoakam and Dale-Chall formulas. By 
adjusting the vocabulary load, the readability levels were interchanged. 
Multiple-choice results based on factual comprehension led McCracken to 



conclude that 



Selections written to confirm with a set of vocabulary 
standards in order to increase or decrease their readability 
actually may not increase or decrease their readability as 
much as indicated. A selection thus written would seem to 
have a contrived or artifi.cial readability level. 

As a secondary purpose, the present study investigated the effective- 
ness of rewritten social studies materials as a means of improving com- 
prehension. Two social studies texts were included in the study. These 
texts contained identical topics and visual aids such as pictures and 
maps, however, the readability levels were different as determined by 
application of the Dale-Chall Readability formula. The easier text 
was rated at a fifth-sixth grade level in readability while the more 
difficult text was placed at the seventh- eighth grade level. A single, 
multiple-choice test was constructed to measure knowledge acquired afyer 
reading a randomly selected chapter. Analysis of covariance was applied 
to factor out the effects of reading achievement levels, I.Q., and 
previous social studies grades. The null hypothesis of no significant 
differences between admusted means was not rejected. 

It is not the intent of this paper to suggest that readability' 
formulas have no validity in adjusting readability levels. However, the 
above findings indicate that attempting to provide more readable materials 
by reducing sentence and vocabulary factors may not benefit the students 
for whom it is intended. 

The Cloze Procedure 

In 1953 Wilson Taylor^^ initiated a completion system which he termed 

the cloze procedure. This system is defined as being a method of 
intercepting a message (written or spoken), mutilating it by deleting parts. 
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and then administering it to receivers (readers or listeners). The 
degree of success in restoring the missing elements is indicative of 
the individual's capacity and/or ability to deal with that message. 

This interaction between the reader and the printed material appears to 
circumvent certain limitations of readability formulas, Taylor^^ suggests 
that the cloze procedure seems to measure the effects of many elements 
of reading by involving the reader with the material to be read. 

Validity and Reliability of the Cloze Procedure 

M^ny studiers have confirmed the validity of the cloze procedure as a 
measure of readability. In his initial experiment, Taylor finds that 
several reading passages were ranked in the same order by the Bale-Chall 
Readability formula, the Plesch Readability formula, and the cloze 
pi^ocedure, Rankin^® reports correlations between standardized reading 
test scores and cloze test scores ranging from .65 to .81. 

A number of studies relate reliability findings for the cloze pro- 
cedure in pre- and post— test scores, TayloiA^ states that such correlations 
for three cloze forms employed in this investigation ranged from .80 to .88. 
Coleman and Miller^O find a correlation of .93 between pre- and post-test 
scores. Hence, the above findings appear to confirm reliability and 
validity of the clcme procedure as a measure of readability. 



Cloze as a Predictor of Comprehension 

As a rationale for this study which investigated the effectiveness of 
the cloze procedure as a predictor of ability to comprehend social studies 
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materials, two studies appeared to be pertinent. Bormuth^l established 
a frame of reference between cloze test scores and equivalent comprehension 
scores. Hafner^^ investigated the effectiveness of the cloze procedure 
as a predictor of course grades in a college methods class with a resultant 
correlation of .65 being reported. These data suggest that the degree 
of comprehensibility an individual finds in instructional material may be 
predicted by pre-reading cloze scores. 



Procedure and Findings 

Data were obtained for this study by the following procedure. Students 
first completed a pre-reading cloze test from one of the two texts utilized 
in the study. An every fifth-word deletion system was employed. After 
completion of the cloze test, the student read the chapter from which the 
cloze test had been constructed and completed a fifty-item multiple-choice 
test. 




To test the hypothesis concerning the predicitve effectiveness of the 
cloze procedure as compared to the predictive effectiveness of the standardiaed 
reading test scores, I.Q. scores, and previous social studies grades as 
predictors of how well students comprehend social studies materials, 
significant differences between tvro correlations coefficients involving 
a common variable were investigated with application of a procedure 
described by Tate^^. At the .01 level the cloze procedure was not found 
to be significantly better than other variables in predicting comprehension 
levifcle. reference to the standardized reading test scores, the findings 
were in the opposite direction of the prediction. At the .05 level, however, 
clo*e scores were found to be significantly better predictors of comprehen- 
sion of the social studies material as measured in this study than I.Q. scores 
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and previous social studies grades. 

Discussion 

A difference in the opposite direction of the prediction was found 
im comparing the effectiveness of prediction of cloze and standardized 
reading test scores. This result might be attributable to the similarity 
of the kinds of questions, i.e., multiple-choice items in the criterion 
measure and the standardized reading test. Completion of the clcze test 
may have required a different, more subjective type of comprehension 
ability than did the standardized reading test. 

The efficacy of rewritten social studies materials on a lower read- 
ability level as a means of improving comprehensibility of such material 
was investigated. Reinforcement was given to certain previous studies 
in that objectively reducing vocabulary difficulty and sentence complexity 
may not significantly improve comprehension scores. 

Continued investigation of the cloze procedure as a predictor of 
comprehension appears to be warrarited. Numerous studies indicate that the 
cloze procedure is a valid and reliable measure of readability. The 
significant differences at the .05 level in comparing the predictive 
effectiveness of cloze scores to I.Q. scores and previous social studies 
grades also support the above suggestion, 

Bormuth's frame of reference was mentioned previously. The findings 
of this study suggest that a universal frame of reference may not be 
feasible. 
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